Fundus imaging

ABSTRACT

Degradations of the image acquired by a fundus camera, including those due to intraocular defects are reduced by digitizing the image, taking an FFT of row and column image data, correlating the FFTs of the rows and of the columns to obtain resultant row product vectors and column vectors. The Nth root of the resultant row and column product vectors is computed, where N is the respective numbers of rows and columns. A minimum offset term is subtracted from each element of the resultant vector to obtain the PSF spatial spectrum (MTF) of the eye. Each row FFT and each column FFT is then divided by the MTF after which the inverse FFT yields a restored distortion-reduced image.

FIELD OF THE INVENTION

[0001] This invention relates to digital signal processing and, moreparticularly, to processing images of the retina to obtain accurateinformation.

BACKGROUND OF THE INVENTION

[0002] While the ophthalmoscope is useful in detecting early signs ofdiabetes, cardiovascular disease and other related conditions, obtaininga wide view of the interior of the eye and the periphery of the retinawith this instrument generally requires that the iris be dilated. A moreconvenient tool for patient screening is provided by the non-mydriaticfundus camera which can be used as a diagnostic tool byophthalmologists, optometrists and other medical professionals.Information is sought, for example, about the condition of the retina'sdendritic pattern of blood vessels, the macula and the optic nerve amongthe main features of the fundus. The non-mydriatic camera attempts tofocus a ring of light through the pupil to illuminate the retina withoutthe need for dilation of the pupil. The ring of light diverges as itexits the posterior crystalline lens of the eye so that a large,defocused spot is formed on the retina. The return optical path betweenthe fundus and the camera again traverses the crystalline lens Theretinal image is acquired by the camera through the central hole in thering of light.

[0003] Unfortunately, the retinal image obtained this way containsseveral major types of distortion. Some of these, such as distortion dueto the camera and its optical system can be compensated by calibrationwith a standard test image. Others, arising in inaccessible portions ofthe optical path, such as intraocular reflections, haze in the vitreoushumor, haze and flare due to a cataractous lens diffusing the peripheralring of light into the central ‘viewing hole’, or cornealinhomogeneities, have not yielded to available image processing methods.In addition to the foregoing, and despite the use of strobed flash,images may be blurred as a result of axial movement of the patient'shead, high velocity, micro-saccadic eye movements, or the operator'sinability simultaneously to achieve all of the conditions necessary fora sharp image. Thus, the very eyes which require the highest level ofdiagnostic image quality (those exhibiting intraocular pathology) arethe ones most likely to suffer from one or more of the foregoing imagedegradation problems. Accordingly, images obtained heretofore byconventional fundus cameras have suffered from an indeterminate amountof distortion and artifactual noise. It would be of inestimable value tobe able to remove such distortions from the image acquired by a funduscamera.

SUMMARY OF THE INVENTION

[0004] We have discovered that, the degradation of the retinal imageacquired by a fundus camera may be considered to be the point spreadfunction (PSF) of the optical path, including the media of the eyeitself (cornea, aqueous humor, crystalline lens, aqueous humor,intraocular membranes, etc.) together with image acquisition apparatus(principally the fundus camera). In accordance with the principles ofthe invention, for processing purposes, this point spread function isdeemed to be constant from point to point throughout the image, or atleast through particular segments thereof. However, the point-to-pointamplitude of the retinal image, which consists largely of dendriticnetworks of blood vessels, varies from point-to-point in a quasi-randommanner. The detected (distorted) image, therefor, consists of therandomly varying ideal image convolved with the relatively constant PSFwhich affects every part of the image in substantially the same way. Theretina, sampled for example in straight lines, such as rows and columns,yields a “random” video signal due to the dendritic fractal nature ofthe retina's anatomy. Any amplitude vector in the image, no matter inwhich direction the image is sliced, e.g., whether vertically orhorizontally, will contain the PSF convolved with the retinal imagedata. Simple inspection of any vector will reveal nothing but randomvariations in amplitude because the convolved PSF has the effect only ofblurring and adding haze to the ideal “random” image features.

[0005] However, if the image is converted from a conventional‘space-domain’ representation to a ‘frequency-domain’ representation,then the spatial-spectrum of the PSF (the modulation-transfer-functionor MTF) of the image may easily be separated from the spatial-spectra ofthe ideal image data, as elucidated below. As is well known (Castleman,et al), a convolution in the space-domain is equivalent to a vectormultiplication in the frequency domain. In accordance with theinvention, the image is first divided into rows and columns and each rowis converted from a space representation to a frequency representationthrough the application of a Fast Fourier Transform. Each transformedvector is stored in computer memory. Each of these vectors contains thespectrum of the row's image data multiplied by the spectrum of the PSF(MTF). If each transformed vector is correlated with every othertransformed vector in the image, the random variation produced by the“ideal” signal will eventually be swamped by the steady value of theMTF. The value of the MTF (or its transform, the PSF), however, is whatwe are looking for and what we need to obtain the ideal image from theimage obtained by the fundus camera. When all of the rows have thus beencorrelated to form a resultant vector, the Nth root of the resultantvector is computed where N is the number of rows that have beenmultiplied. The resultant vector now contains the mean MTF of theoptical system. Each element of this mean MTF vector is offset by a moreor less constant DC term. This term is the product of all of the randomamplitudes of the ideal image data of each column. We have found, inpractice, that the minimum value in the result vector may now be locatedand subtracted from all of the elements of the result vector to yieldthe correct MTF values. However, in some special cases, coefficients maybe applied to the offset data to yield a more accurate estimate of theMTF. Dividing each FFT-transformed row by the MTF and obtaining theinverse FFT of the rows yields the horizontally restored ideal image.The procedure may then be repeated by similarly correlating all of thecolumn vectors to yield the vertically restored ideal image.

BRIEF DESCRIPTION OF THE DRAWING

[0006] The foregoing objects and features of the invention may becomemore apparent from the ensuing written description when read togetherwith the drawing in which:

[0007]FIG. 1 is a flow chart of the process of the invention.

DETAILED DESCRIPTION

[0008] Before referring to FIG. 1, an overall review of the process ofthe invention may be helpful. Let I represent the object of interest,e.g., the retina of the human eye. Let E represent the point spreadfunction (PSF) of the media of the eye and let the PSF of a camera suchas a fundus camera (not shown) and imaging optics be represented by C.Then the actual image A reported by the camera is the retinal object (i)degraded by being convolved with E and C. In the frequency domain, aconvolution is a simple vector multiplication. Therefore, a fast Fouriertransform (FFT) taken of the actual image A is the product of the FFTsof I, E and C. If, for the moment, we accept that the PSF of a highquality fundus camera makes a negligible contribution to the image thenthe FFT of A can be assumed to be the product of the FFTs of I and E.So, to improve A, we must divide its Fourier transform by the Fouriertransform of E. In the simplest case, E should remain constantthroughout the media of the eye. Let us scan and digitize the imageobtained by the fundus camera. For example, let the image be scanned byrows and columns. In our acquired discrete digital image therefore, Ehas been convolved with every line and row of the image A. Now, let usconsider the retinal object, I, which contains blood vessels and nervesarranged in 2-dimensional dendritic fractal-like pseudo-randomstructures. Therefore: 1-dimensional column/row samples of I areessentially random waveforms. But, of course₇ the random waveforms areall convolved with E. So, if we correlate all of the rows, we will get aresult row vector which is the constant E sitting on top of a “DC”offset which represents all of the randomness of the ideal image, I. Wedo the same for the columns of the image and measure its DC offset andsubtract it, leaving the PSF of the EYE itself. Then we divide each rowand column of A by E and we do an inverse FFT and obtain a corrected andclear image.

[0009] Referring now to FIG. 1, at step 100 the image acquired by thefundus camera is digitized. At step 101 an FFT is taken of the “next”row of the digitized image data. The first time the process is executedthe “next” row is, of course, the first row. At step 102 a determinationis made whether the last row of the digitized data has been FFT'd. Atstep 103, if the last row had been reached in step 102, the row vectorof FFT data is copied to the FFT result vector. In step 105 the resultvector is multiplied by the next FFT row vector. This is the correlationstep discussed above. In step 107 a determination is made whether all ofthe FFT row vectors have been correlated. The process continues withstep 110 which computes the Nth root of the FFT result vector, where Nis the number of rows. In step 112 a minimum offset is subtracted fromall elements of the FFT result vector. In step 114 each FFT row vectoris divided by the FFT result vector. In step 116 the inverse FFT istaken and in step 200 the foregoing procedure is repeated for eachcolumn of the acquired image data. The inverse FFT produces a correctedimage.

[0010] What has been described is deemed to be illustrative of theprinciples of the invention. It will be apparent, for example, to thoseskilled in the art that the method of the present invention may beapplied to restore images acquired in other ways (e.g., ultrasound, CTscans, snapshots, satellite photos, etc.) so long as the image does notcontain appreciable amounts of periodic data. Transforms other than theFFT may advantageously be used. For example, the Z-Transform may ingeneral be used to take discrete signal amplitudes (such as thoseobtained by sampling an image of the retina taken by a fundus camera)into a complex-variable domain where it plays a similar role to the onethat the Laplace Transform does in the continuous time domain. Like theLaplace transform, the Z-transform offers a different way solvingproblems and designing discrete domain applications. It will also beapparent that the scanning or sampling of the image may be done radiallyfrom any desired center of interest, such as the center of sight, thefovea, the macula, etc., and that non-linear scans may also be employedso that more samples are taken at areas of interest. It may also beuseful, prior to the application of the restoration method describedabove, to re-sample images at a higher resolution than the original andthen low-pass filter the result. Subjecting the images to such“anti-aliasing”, prior to computing the MTF, may produce a result thatis useful in some applications. Further and other modifications will beapparent to those skilled in the art and may be made without, however,departing from the spirit and scope of the invention.

What is claimed is:
 1. Removing from a retinal image acquired by afundus camera, image degradations arising from intraocular defects,comprising the steps of: a.) digitizing said acquired image; b.) takingan FFT of said digitized image by rows and columns; c.) correlating saidFFTs to obtain resultant row product vectors and column vectors; d.)finding the root equal to the respective numbers of rows and columns ofthe resultant row and column product vectors to obtain quotients; e.)subtracting from each of said quotients a minimum offset term to obtainthe PSF spatial spectrum (MTF) of the eye; f.) dividing each row FFT andeach column FFT by said MTF; and g.) taking the inverse FFT to yield arestored distortion-reduced image.
 2. The method of removing from anacquired image degradations arising from optical defects in inaccessibleportions of the optical path, comprising the steps of a.) digitizingsaid acquired image; b.) scanning said acquired image alongpredetermined paths to obtain vectors of data; c.) taking a discretetransform of said vectors of data; d.) correlating said discretetransform of said vectors to obtain resultant product vectors; e.)finding roots of the resultant product vectors for each of saidpredetermined paths; f.) subtracting from each of said roots a minimumoffset term to obtain a point spread function spatial spectrum (MTF);g.) dividing each discrete transform of said vectors by said MTF; andh.) taking the inverse discrete transform to yield a restoreddistortion-reduced image.
 3. The method of claim 2 wherein said acquiredimage is a retinal image acquired by a fundus camera.
 4. The method ofclaim 3 wherein at least one of said predetermined paths traverses apredetermined feature of said retinal image.
 5. The method of claim 3wherein said discrete transform is a fast Fourier transform.
 6. Themethod of claim 4 wherein said predetermined paths are row and columnpaths of said image.
 7. The method of claim 5 wherein there are N ofsaid predetermined paths and said root is the Nth root of said productvectors.